1. Field of Invention
This invention is directed to a document organizing system. In particular, this invention is directed to a method and a system for organizing documents based upon the context of annotations made to those documents.
2. Description of Related Art
When people read paper documents, they often make annotations to highlight interesting or controversial passages and to record their reactions. Common annotations include margin notes, vertical bars, stars, circles, underlines, highlights, etc. Two advantages of annotating directly on the page are its low overhead and convenience. One disadvantage is that the recorded information is hidden and inaccessible until the reader returns to the specific page in the specific document.
To avoid this problem, some readers use a separate reading notebook to record their annotations. A reading notebook is useful because it provides a separate summary of what the user has read along with any commentary. The advantage of a reading notebook is that it permits a quick review of the material because it generally has less information to browse and search than the original document. One disadvantage of a reading notebook, however, is that the reader must recreate the context for each note to fully understand the meaning of each note.
Readers also use note cards to organize notes. The advantage of a note card system is that the cards can be easily reorganized. However, as with a reading notebook, unless the reader recreates it, there is no context available to permit the user to fully understand the notes. Additionally, each note must be categorized onto the correct card before it can be recorded.
Handwritten notes and keywords are used in a system known as "Marquee" to index video. This system is described in "Marquee: A Tool for Real-Time Video Logging", K. Webber et al., Proceedings of CHI '94, April 1994, pp. 58-64, incorporated herein by reference in its entirety. In "Marquee", notes are synchronized to a video string with time zones that are created with horizontal line gestures. Keywords are identified by the user by circling the words and notes that the user has selected as keywords. The keywords are assigned to the time zone in which the keyword is created. Keywords also may be assigned directly by the user by typing the keyword in manually. Because the keywords are associated with time the user can view an index of time zones and go directly to the video by selecting a time zone using an index of the previously identified keyword or annotations. Although "Marquee" uses annotations to index a video document, it does not combine the annotations with the document in a visual way. "Marquee" is thus analogous to notetaking in a separate notebook rather than on the document itself.
"Dynomite" is a free-form digital "ink" notebook. The digital ink notebook is a pen-based computer that the user controls by writing with a pen directly on the screen of the computer. The computer senses the location and the positions traversed as the pen moves across the display and assigns ink marks that correspond with the positions of the pen. These ink marks are called digital ink because the ink is described by the computer digitally. Dynomite extracts the ink, assigns properties to each ink mark and can present a list of the ink marks sorted by the assigned properties. This list is known as an ink index. This system is described in co-assigned and co-pending U.S. patent application, Ser. No. 08/821,311, filed on Mar. 20, 1997, entitled "System for Capturing and Retrieving Audio Data and Corresponding Handwritten Notes", and "Dynomite: A Dynamically Organized Ink and Audio Notebook", by L. Wilcox et al., In CHI '97 Conference Proceedings, ACM Press, 1997, pp. 186-193, incorporated herein by reference in their entireties. This ink index shows a "type" of the "ink" along with a time stamp and provides links to the original notebook pages. Dynomite's ink index provides "ink" marks linked to the corresponding full notebook page. However, Dynomite organizes only the ink notes themselves and not the associated information.
"ComMentor" is a platform for shared annotations that attaches text-based comments to locations within web documents. This system is described in "Shared Web Annotations as a Platform for Third-Party Value-Added Information Providers: Architecture, Protocols, and Usage Examples", by M. Roscheisen, et al., Technical Report STAN-CS-TR-97-1582, Stanford Integrated Digital Library Project, Computer Science Department, Stanford University, November 1994, Updated April 1995, incorporated herein by reference in its entirety. Annotations are grouped into sets. A user can filter these sets and tour through documents within a set. A tour window shows a list of annotations, each annotation shown with the document title of the annotated document and a number of annotation attributes. Clicking on the annotation causes the display to jump to the source document at the position of the annotation. ComMentor uses filtered annotations to produce lists of read documents, but does not support paper-like annotations or present lists of annotations in context.
Classroom 2000 is a system for capturing a lecture using recorded audio, prepared visual materials and handwritten notes made on a display overlay of viewgraphs. This system is described in "Classroom 2000: Enhancing Classroom Interaction and Review", by G. Abowd et al., In Proceedings of CSCW '96, March 1996, incorporated herein by reference in its entirety. Searching the text in the viewgraphs retrieves the viewgraphs along with the overlaid notes.
The Freestyle system, which was developed at Wang Laboratories, is a mechanism for sketching and writing on screen snapshots or on sheets of electronic paper. Freestyle records cursor movement and audio as well as the handwriting. This system is described in "Rapid Integrated Design of a Multimedia Communication System, and Human-Computer Interface Design", E. Francik, Marianne Rudisill et al. (editor), Morgan Kaufman Publishers, Inc.,1996, incorporated herein by reference in its entirety. The result is a dynamic multimedia message that can be mailed to others. Freestyle does not provide the ability to organize the handwritten annotations.
The PENPOINT operating system for pen-based computers, recognizes pen gestures for editing and allows arbitrary "ink" marks to be placed on top of any document using an "acetate layer". This system is described in "The Power of PENPOINT", by R. Carr et al., Addison-Wesley, Inc., 1991, incorporated herein by reference in its entirety. Although both Freestyle and PENPOINT support free-form document annotation, neither provides any way to retrieve documents based upon those annotations.
In 1945 Vannevar Bush described a vision of a mesh of trails running through a mechanized private file and library or memex in "As We May Think", in Atlantic Monthly, July 1945, pp. 101-108, incorporated herein by reference in its entirety. These trails were produced as part of the reading activity, and provided a way to create and share personal organizations of information. Bush's visions were seminal in the development of hypermedia systems such as Engelbart's NLS and the World Wide Web. However, hypermedia systems have focused on sharing, browsing and more explicit authoring of links, not on personal organization and annotation.
Thus, an annotation system for electronic documents is needed that combines the advantages of marking directly on a document with quick accessibility and the flexible organization of marking on note cards or in a notebook.